Web Survey Bibliography
Title Non-response in evaluation of teaching
Author Brinkmoeller, B.; Forthmann, B.; Thielsch, M.
Year 2016
Access date 29.04.2016
Presentation PDF (527KB)
Abstract
Relevance & Research Question
Student evaluations of teaching (SET) gain more and more importance for university teaching, especially web-based SETs. To guarantee the validity of SET and to avoid sample-errors high response rates are necessary – which cannot always be reached. This study investigates different factors, which might give some explanations for nonresponse in SETs. We examined the social exchange-theory, salience, opportunity costs, survey fatigue and the survey mode (online vs. paper-pencil).
Methods and data
We contacted student representatives of 48 universities, additional student groups in social media, and sent approx. 900 invitations via the online-panel PsyWeb. The web-based survey was available for about four months. Participants provided reasons for non-response, information about response behavior, and attitudes. A total of 490 participants (69.59 % female; age: M = 24.10, SD = 4.07) were included in the final sample.
Results
First, our results show a significant influence of social exchange on responding in SET (Δ R²=.069, p < .001). Also the influence of salience (ΔR²=.201, p < .001) and survey fatigue (ΔR²= .078, p < .001) show significant influences on the participation in SETs. No significant effects were found for opportunity costs (ΔR² = .005, p = .166) as well as for the survey mode (ΔR²= .004, p = .209).
Added value
The results of our study can be helpful for online researchers and evaluation managers in reducing non-response. Notably, our findings stress the importance of communication between students: It influences a student’s evaluation behavior if a fellow student evaluates all of his or her lectures and courses. Thus, universities should indicate how many students take part in a current SET to motivate even more students. Furthermore, it is helpful to increase the students’ identification with their own university and SET, for example with special events or university-games. In addition, the consequences of the SET should be public for the students so they become aware of how they can influence the quality of teaching in their faculty. Finally, we could find no evidence for the often made assumption that an online administration of questionnaires leads to a non-response-problem in SETs.
Student evaluations of teaching (SET) gain more and more importance for university teaching, especially web-based SETs. To guarantee the validity of SET and to avoid sample-errors high response rates are necessary – which cannot always be reached. This study investigates different factors, which might give some explanations for nonresponse in SETs. We examined the social exchange-theory, salience, opportunity costs, survey fatigue and the survey mode (online vs. paper-pencil).
Methods and data
We contacted student representatives of 48 universities, additional student groups in social media, and sent approx. 900 invitations via the online-panel PsyWeb. The web-based survey was available for about four months. Participants provided reasons for non-response, information about response behavior, and attitudes. A total of 490 participants (69.59 % female; age: M = 24.10, SD = 4.07) were included in the final sample.
Results
First, our results show a significant influence of social exchange on responding in SET (Δ R²=.069, p < .001). Also the influence of salience (ΔR²=.201, p < .001) and survey fatigue (ΔR²= .078, p < .001) show significant influences on the participation in SETs. No significant effects were found for opportunity costs (ΔR² = .005, p = .166) as well as for the survey mode (ΔR²= .004, p = .209).
Added value
The results of our study can be helpful for online researchers and evaluation managers in reducing non-response. Notably, our findings stress the importance of communication between students: It influences a student’s evaluation behavior if a fellow student evaluates all of his or her lectures and courses. Thus, universities should indicate how many students take part in a current SET to motivate even more students. Furthermore, it is helpful to increase the students’ identification with their own university and SET, for example with special events or university-games. In addition, the consequences of the SET should be public for the students so they become aware of how they can influence the quality of teaching in their faculty. Finally, we could find no evidence for the often made assumption that an online administration of questionnaires leads to a non-response-problem in SETs.
Access/Direct link Conference Homepage (presentation)
Year of publication2016
Bibliographic typeConferences, workshops, tutorials, presentations
Web survey bibliography - 2016 (264)
- Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems; 2017; Romano, M. F.; Sardella, M. V.; Alboni, F.
- Socially Desirable Responding in Web-Based Questionnaires: A Meta-Analytic Review of the Candor Hypothesis...; 2016; Gnambs, T.; Kaspar, K.
- Dynamic Question Ordering in Online Surveys; 2016; Early, K.; Mankoff, J.; Fienberg, S. E.
- How to use online surveys to understand human behaviour concerning window opening in terms of building...; 2016; Fabbri, K.
- Impact of satisficing behavior in online surveys on consumer preference and welfare estimates; 2016; Gao, Z.; House, L. A.; Bi, X.
- Comparing Twitter and Online Panels for Survey Recruitment of E-Cigarette Users and Smokers; 2016; Guillory, J.; Kim, A.; Murphy, J.; Bradfield, B.; Nonnemaker, J.; Hsieh, Y. P.
- Influence of Importance Statements and Box Size on Response Rate and Response Quality of Open-Ended...; 2016; Kumar Chaudhary, A.; Israel, G. D.
- Web based health surveys: Using a Two Step Heckman model to examine their potential for population health...; 2016; Morrissey, K.; Kinderman, P.; Pontin, E.; Tai, S.; Schwannauer, M.
- “Better do not touch” and other superstitions concerning melanoma: the cross-sectional web...; 2016; Gajda, M.; Kamiñska-Winciorek, G.; Wydmañski, J.; Tukiendorf, A.
- Methods for Evaluating Respondent Attrition in Web-Based Surveys; 2016; Hochheimer, C. J.; Sabo, R. T.; Krist, A. H.; Day, T.; Cyrus, J.; Woolf, S. H.
- The Low Response Score (LRS): A Metric to Locate, Predict, and Manage Hard-to-Survey Populations; 2016; Erdman, C.; Bates, N.
- Targeted Appeals for Participation in Letters to Panel Survey Members; 2016; Lynn, P.
- Can we assess representativeness of cross-national surveys using the education variable?; 2016; Ortmanns, V.; Schneider, S.
- Methodological Aspects of Central Left-Right Scale Placement in a Cross-national Perspective; 2016; Scholz, E.; Zuell, C.
- Fieldwork Effort, Response Rate, and the Distribution of Survey Outcomes: A Multilevel Meta-analysis; 2016; Sturgis, P.; Williams, Jo.; Brunton-Smith, I.; Moore, J.
- Mobile-only web survey respondents; 2016; Lugtig, P. J.; Toepoel, V.; Amin, A.
- Comparison of Face-to-Face and Web Surveys on the Topic of Homosexual Rights; 2016; Liu, M.; Wang, Yic.
- Question order sensitivity of subjective well-being measures: focus on life satisfaction, self-rated...; 2016; Lee, S.; McClain, C.; Webster, N.; Han, S.
- Web-Based Statistical Sampling and Analysis; 2016; Quinn, A.; Larson, K.
- Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys 2016; 2016
- Using Visual Analogue Scales in eHealth: Non-Response Effects in a Lifestyle Intervention; 2016; Kuhlmann, T.; Reips, U.-D.; Wienert, J.; Lippke, S.
- Development and Pilot Test of a Mobile Application for Field Data Collection; 2016; Chiappetta, L.; Kerr, M. M.
- Statistical Design for Online Experiments Across Desktops, Tablets, Smartphones (and Maybe Wearable...; 2016; Qian, P.; Sadeghi, S.; Arora, N. K.
- A Case Study on the Use of Propensity Score Adjustments with Web Survey Data; 2016; Parsons, V.
- Motivated Misreporting in Web Panels; 2016; Bach, R.; Eckman, S.
- Are Initial Respondents Different from the Nonresponse Follow-Up Cases? A Study of Probability-Based...; 2016; Zeng, W.; Dennis, J. M.
- Using official surveys to reduce bias of estimates from nonrandom samples collected by web surveys; 2016; Beresovsky, V.; Dorfman, A.; Rumcheva, P.
- Predicting and Preventing Break-Offs in Web Surveys; 2016; Mittereder, F.
- A Feasibility Study of Recruiting and Maintaining a Web Panel of People with Disabilities; 2016; Chandler, J.
- Exploration of Methods for Blending Unconventional Samples with Traditional Probability Samples; 2016; Gellar, J.; Zhou, H.; D.; Sinclair, M. D.
- Ratio of Vector Lengths as an Indicator of Sample Representativeness ; 2016; Shin, H. C.
- Design of Sample Surveys That Complement Observational Data to Achieve Population Coverage; 2016; Slud, E.; Ashmead, R.
- Inferences from Internet Panel Studies and Comparisons with Probability Samples; 2016; Lachan, R.; Boyle, J.; Harding, R.
- Exploring the Gig Economy Using a Web-Based Survey: Measuring the Online 'and' Offline Side...; 2016; Robles, B. J.; McGee, M.
- Comparing data quality between online panel and intercept samples; 2016; Liu, M.
- Effect of a Pre-Paid Incentive on Response Rates to an Address-Based Sampling (ABS) Web-Mail Survey; 2016; Suzer-Gurtekin, Z.; Elkasabi, M.; Liu, Me.; Lepkowski, J. M.; Curtin, R.; McBee, R.
- Response Behavior in a Video-Web Survey: A Mode Comparison Study; 2016; Haan, M.; Ongena, Y. P.; Vannieuwenhuyze, J. T. A.; de Glopper, K.
- Standard Definitions Final Dispositions of Case Codes and Outcome Rates for Surveys; 2016
- Integration of a phone-based household travel survey and a web-based student travel survey; 2016; Verreault, H.; Morency, C.
- Evaluation of mode equivalence of the MSKCC Bowel Function Instrument, LASA Quality of Life, and Subjective...; 2016; Bennett, A. V.; Keenoy, K.; Shouery, M.; Basch, E.; Temple, L. K.
- Making use of Internet interactivity to propose a dynamic presentation of web questionnaires; 2016; Revilla, M.; Ochoa, C.; Turbina, A.
- A streamlined approach to online linguistic surveys; 2016; Erlewine, M. Y.; Kotek, H.
- Du kommst hier nicht rein: Türsteherfragen identifizieren nachlässige Teilnehmer in Online-Umfragen; 2016; Merkle, B.; Kaczmirek, L.; Hellwig, O.
- Incorporating eye tracking into cognitive interviewing to pretest survey questions; 2016; Neuert, C.; Lenzner, T.
- Population Survey Features and Response Rates: A Randomized Experiment; 2016; Guo, Y.; Kopec, J.; Cibere, J.; Li, L. C.; Goldsmith, C. H.
- Mode Effect and Response Rate Issues in Mixed-Mode Survey Research: Implications for Recreational Fisheries...; 2016; Wallen, K. E.; Landon, A. C.; Kyle, G. T.; Schuett, M. A.; Leitz, J.; Kurzawski, K.
- A measure of survey mode differences; 2016; Homola, J.; Jackson, N. M.; Gill, Je.
- Web Health Monitoring Survey: A New Approach to Enhance the Effectiveness of Telemedicine Systems ; 2016; Romano, M. F.; Sardella, M. V.; Alboni, F.
- Smartphones vs PCs: Does the Device Affect the Web Survey Experience and the Measurement Error for...; 2016; Toninelli, D.; Revilla, M.
- Question order sensitivity of subjective well-being measures: focus on life satisfaction, self-rated...; 2016; Lee, S.; McClain, C.; Webster, N.; Han, S.